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Abstract 

In the paper we study asymptotic properties of the adaptive pro- 
cedure proposed in the paper Galtchouk, Pergamenshchikov, 2007, 
for nonparametric estimation of unknown regression. We prove that 
this procedure is asymptotically efficient for some quadratic risk, i.e. 
we show that the asymptotic quadratic risk for this procedure coin- 
cides with the Pinsker constant which gives a sharp lower bound for 
quadratic risk over all possible estimates. 
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1 Introduction 



The paper deals with the estimation problem in the heteroscedastic non- 
parametic regression model 

yj = S{x,)+a,{S)Cj, (1.1) 

where the design points xj = j/n, S{-) is an unknown function to be esti- 
mated, {S,j)i<j<n is a sequence of centered i.i.d. random variables with unit 
variance and E^^ = ^* < oo, {(^j{S))i^j^j^ are unknown scale functionals 
depending on unknown regression function S and the design points. 

Typically, the notion of asymptotic optimality is associated with the 
optimal convergence rate of the minimax risk (see for example, Ibragimov, 
Hasminskii,1981; Stone, 1982). An important question in optimality results 
is to study the exact asymptotic behaviour of the minimax risk. Such results 
have been obtained only in a limited number of investigations. As to the 
nonparametric estimation problem for heteroscedastic regression models we 
should mention the papers Efromovich, 2007, Efromovich, Pinsker, 1996, 
and Galtchouk, Pergamenshchikov, 2005, concerning the exact asymptotic 
behaviour of the £2"^isk and paper by Brua, 2007, devoted to the efficient 
pointwise estimation for heteroscedastic regressions. 

We remind that an example of heteroscedastic regression models is given 
by econometrics (see, for example, Goldfeld, Quandt, 1972, p. 83), where 
for consumer budget problems one uses some parametric version of model 
(jl.ip with the scale coefficients defined as 

a]{S) = Co + c^x^+C2S\xj), (1.2) 
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where Cg, and C2 are some positive unknown constants. 

The purpose of the article is to study asymptotic properties of the adap- 
tive estimation procedure proposed in Gahchouk, Pergamenshchikov, 2007, 
for which a non-asymptotic oracle inequality was proved for quadratic risks. 
We will prove that this oracle inequality is asymptotically sharp, i.e. the 
asymptotic quadratic risk is minimal. It means the adaptive estimation 
procedure is efficient under some conditions on the scales (o"j(S'))x<j<„ 
which are satisfied in the case (jl.2p . Note that in Efromovich, 2007, Efro- 
movich, Pinsker, 1996, an efficient adaptive procedure is constructed for 
heteroscedastic regression when the scale coefficient is independent of S, i.e. 
(7j{S) = (Tj. In Galtchouk, Pergamenshchikov, 2005, for the model (jl.ip 
the asymptotic efficiency was proved under strong conditions on the scales 
which are not satisfied in the case ()1.2p . Moreover in the cited papers the 
efficiency was proved for the gaussian random variables (Cj)i<j<„ that is 
very restrictive for applications of proposed methods to practical problems. 

In the paper we modify the risk by introducing into a additional supre- 
mum with respect to a classe of unknown noise distributions like to Galtchouk, 
Pergamenshchikov, 2006. This modification allow us to eliminate from the 
risk dependence on the noise distribution. Moreover for this risk a efficient 
procedure is robust with respect to changing of noise distributions. 

It is well known to prove the asymptotic efficiency one has to show 
that the asymptotic quadratic risk coincides with the lower bound which 
is equal to the Pinsker constant. In the paper two problems are resolved: 
in the first one an upper bound for the risk is obtained by making use of 
the non-asymptotic oracle inequality from Galtchouk, Pergamenshchikov, 
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2007, in the second one we prove that this upper bound coincides with 
the Pinsker constant. Let us remind that the adaptive procedure proposed 
in Galtchouk, Pergamenshchikov, 2007, is based on weighted mean-squares 
estimates, where the weights are corresponding modifications of the Pinsker 
weights for the homogene case (when cri{S) = . . . = (T„(S') = 1) relative to a 
certain smoothness of the function S and this procedure chooses an estimator 
best for the quadratic risk among these estimates. To obtain the Pinsker 
constant for the model (jl.ip one has to prove a sharp asymptotic lower 
bound for the quadratic risk in the case when the noise variance depends on 
the unknown regression function. This lower bound is obtained by making 
use of an inequality of kind of the van Trees inequality (see. Gill, Levit, 
1995). First we prove the inequality for a parametric regression with the 
noise variance depending on the unknown regression (see Section 6) and 
further we apply the inequality to the nonparametric regression by standard 
reducing to a parametric case. 

The paper is organized as follows. In Section [2] we construct a adaptive 
estimation procedure. In Section [3] we formulate principal conditions. The 
main result is given in Section HI The upper bound for the quadratic risk is 
given in Section [5j In Section [6] we find the lower bound for a parametric 
model. In Section [7| we study the parametric family. In Section [8] we obtain 
the lower bound for model (|l.ip . An appendix contains some technical 
results. 
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2 Adaptive procedure 

In this section we describe the adaptive procedure proposed in We make 
use of the standard trigonometric basis {4'j)j>i iii ^2[0, 1], i.e. 

Mx) = 1 , </'j(x) = V2Trj{2TT[j/2]x) , j>2, (2.1) 

where the function Trj{x) = cos(x) for even j and Trj{x) = sin(x) for 
odd j; [x] denotes the integer part of x. We remind that if n is odd then 
the functions i4'j)i<j<n orthonormal with respect to the empirical inner 
product generated by the sieve {xj)i^j<^n in (jl.lh . i.e. for any 1 < i,j < n, 

1 " 
1=1 

where Kr^^ is Kronecker's symbol. Thanks to this basis we pass to the 
discrete Fourier transformation of model (jl.ip . i.e. 

^,,n = ^,,n + (l/V^K,,n, (2.2) 

where e.j^^ = {Y, (f)j)n, 9j^^ = {S, (j)j)n and 

1 " 
1=1 

Here Y = (y^^, . . . ,y„)' and S = {S{xi), . . . ,S'(x^))'. The prime denotes the 
transposition. 

We estimate the function S by the weighted least squares estimator 

n 

Sx = Y.^(^An^j^ (2-3) 
i=i 

where the weight vector A = (A(l), . . . , A(n))' belongs to some finite set A 
from [0,1]" with n > 3. Here we make use of the weight family A introduced 
in [B], i.e. 



A = {\^,aeA,}, A, = {l,...,h}x{ti,...,tm}, (2.4) 

where A;* = U =i£, m= and e = 1/lnn. 

For any a = {(3, t) G we define the weight vector = (Aq,(1), . . . , X^in))' 

as 

KU) = hi<j<jo} + - ioo<i<-(a)} , (2.5) 

where jo = jo{a) = [Lo{a)/lnn], u{a) 

= (^^i)V(2/?+i)nV{2/3+i) and 
^^ = (/3+l)(2/3 + l)/(7r2/3/3). 

To find the optimal weights we choose the cost function equals to the pe- 
nalized mean integrated squared error in which unknown parameters are 
replaced by some estimators. The cost function is as follows 

n n 
JnW = E - 2 E ^(•^■) + , (2-6) 



where 



1 " 

hn = ^ln--^n with ^„ = J] (2.7) 

and = [n^/^ + 1] . The penalty term we define as 

4(A) = |A|-t^^O) .= 3^. 

for some 7 > 0. Finally, we set 

A = argmin^g^ J„(A) and S^ = S-^. (2.8) 

The goal of this paper is to study asymptotic (n — > 00) properties of this 
estimation procedure. 



3 Conditions 



First we impose some conditions on unknown function S in model (jl.ip . 

> M func- 
tions. We assume that S belongs to the following set 

k 



Let Cpg^-^(R) be the set of 1-periodic k times differentiable 



j=Q 



where || • || denotes the norm in £2[0i !]> i-^- 



(3.1) 



fmt. (3.2) 

Moreover, we suppose that r > and k > 1 are unknown parameters. 
Note that, we can represent the set W!^ as an ellipse in £2[0) 1]; i-^- 



where 



and 



^. = {f,(^.)= / /(t)0^.(t)dt 

•^0 



«=0 



i=0 



(3.3) 



(3.4) 



(3.5) 



Here {4'j)j>i is the trigonometric basis defined in (j2.ip . 

Now we decribe the conditions on the scale coefficients {aj{S))j^i. 



Hi) aj{S) = g{xj, S) for some unknown function g : [0, 1] x Li[0, 1] 
which is square integrable with respect to x such that 



lim sup 



n 



-'^gHxj,S)-,{S) 



0. 



(3.6) 
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where 'j(S') := g^{x, S)dx. Moreover, 



and 



g,= inf M g\x,S) >0 (3.7) 
o<a;<i sew^ 



sup ^{S) < oo . (3.8) 



H2) For any x G [0, 1] the operator g'^{x, •) : C[0, 1] — *■ R is differentiable 
in the Frechet sense for any fixed function /q from C [0, 1] , i. e. for 
any f from some vicinity of /g in C [0, 1] 

g'^ix, f) = g\x, /o) + l.^j^{f - /o) + T{x, f„ f) , 

where the Frechet derivative L^; : C [0, 1] — ^ R is a bounded linear 
operator and the residual term T(x,/o,/) for each x G [0,1] satisfies 
the following property 

l/-/oU-»o I/-/0I* 
where \f\^ = suPq<^<-^ \f{t)\. 

H3) There exists some positive constant C* such that for any function S 
from C[0, 1] the operator L^, 5 defined in condition H2) satisfies the 
following inequality for any function f from C[0, 1] 

|L.,5(/)I < C* {\S{x)f{x)\ + \f\, + ||S|| 11/11) , (3.9) 

where \f\,=J^\f{t)\dt. 

H4) The function 5o(') ~ y'^i'i'^o) corresponding to Sq = is continuous 
on the interval [0, 1] . Moreover, 

lim sup sup \g'^{x,S) — g'^{x,SQ)\ = 0. 

Q<x<l \S\^<S 
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Now we give some examples of functions satisfying conditions H^)-H4). 

We fix some Cg > 0. Let G : [0, 1] x R — > [cg , +oo) be a function such 
that 



hm max sup\G{u,y) — G{v,y)\ = . (3.10) 

5— >0 |u— 1;|<(5 , 



G'^= sup sup|G (x,y)|/|y| < cx). (3.11) 

0<a:<l- 



and 



Moreover, let y : M ^ be a continuously differentiable function such 
that 

< = sup \V{y)\/{l + \y\) < oo. 

ym 

We set 

r-l 

2 



g\x,S) = G{x,S{x))+ I V{S{t))dt. (3.12) 

In this case 







^{S)= Git,S{t))dt+ [\{S{t))dt 

and for any S G 



'0 

rk 



n 

n-'Y.g'{xj,S)-,{S) 



r \Gixj,S{x^))-G{t,S{t))\dt 
j=i ■^^■j-i 

<A„ + ^ f' \S{t)\\S{t)\dt<A^ + ^r, 
n Jq n 



where 



A„ = max sup \G{u,y) - G{v,y) \ . 

\u—v\<l/n , 



Therefore by condition (jS.lOp we obtain H^). 

Moreover, the Frechet derivative in this case is given by 

Ksif) = Gy{x, S{x))f{x) + C V{S{t))f{t)dt . 

Jn 
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It is easy to see that this operator satisfies the inequahty (|3.9p with 

C* = G' +v' . 

For example, we can take in (j3.12p 

G{x, y) = Cq + CiX + C2y^ and V{x) = c^x'^ (3.13) 

with some coefficients Cq > 0, q > 0, z = 1,2,3. Therefore, we obtain the 
function ([OD if we put in (IXT2D - (l3T3D Cg = 0, i.e. V = 0. 

4 Main results 

Denote by V^, the family of unknown noise density. Remind that the noise 
random variables (Cj)i<j<„ are centered with unit variance and E^^ < ^*, 
where ^* > 3. For any estimate S we define the following quadratic risk 

7^„(5,S)= supEs,p||5-5||^, (4.1) 
pen 

where E^^ is the expectation with respect to the distribution P^^ of the 
observations (y^, . . . ,y„) with the fixed function S and the fixed density p 
of random variables (Cj)i<j<n in model (jl.ip . \\S\\'^ = {S,S)^. 

In Galtchouk, Pergamenshchikov, 2007, we shown the following non- 
asymptotic Oracle inequality for procedure (j2.8p . 

Theorem 4.1. Assume that in model (jl.ip the function S belongs to W\ 
Then, for any odd n > 3, any < p < 1/3 and r > 0, the estimate 
satisfies the following oracle inequality 

n^iS,,S) < (l + K(p))min7^„(5„5) +n-'BM, (4-2) 

AeA 
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where 

K{p) = (6p - 2p2)/(i _ 3p) 
and the function B^{p) is such that, for any 6 > 0, 

lim BM/n' = 0. (4.3) 

n— »oo 

Now we formulate the main asymptotic results. To this end for any 
function S G Wj^ we set 

7fc(5) = Tl ri/(2^'+i) , (4.4) 

where 

n = {2k + l)i/(2*^-+i) (A;/(vr (A; + l)))2'=/(2'=+i) . 

It is well known (see, for example, Nussbaum, 1985) that for any function 
5 G the optimal convergence rate is 77,2^/(2^+1)^ 

Theorem 4.2. Assume that in model (jl.ip t/ie sequence {aj{S)) fulfils the 
condition Hi). T/ien the estimator /rom (|2.8p satisfies the inequality 

limsup n2^'/(2^'+i) sup 7^„(^,, 5)/7fc(5) < 1 . (4.5) 

r 

The following result gives the sharp lower bound for risk (j4.ip and show 
that 7fc(S') is the Pinsker constant. 

Theorem 4.3. Assume that in model (jl.ip the sequence {aj{S)) satisfies 
the conditions H2)- H4). Then, for any estimate Sn, the risk TZn{Sn,S) 
admits the following asymptotic lower bound 

liminf inf sup 7^„(Sn, 5)/7fc(5) > 1 . (4.6) 

r 
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Remark 4.1. Note that in Galtchouk, Pergamenshchikov, 2005 an asymp- 
totically efficient estimate was constructed and results similar to Theorems 
\4-2\ and \4-cl\ were claimed for the model In fact the upper bound is true 

there under some additional condition on the smoothness of the function S, 
i.e. on the parameter k. In the cited paper this additional condition is not 
formulated since erroneous inequality {A.6). To avoid the use of this in- 
equality we modify the estimating procedure by introducing the penalty term 
pP^{\) in the cost function ()2.6p . By this way we remove all additional 
conditions on the smoothness parameter k. 

5 Upper bound 

In this section we prove Theorem 14.21 To this end we wih make use of oracle 
inequahty ()4.2p . We have to find an estimator from the family ()2.3p - (|2.4p for 
which we can show the upper bound (j4.5p . We start with the construction 
of such an estimator. First we put 

= inf{i >l : ie> r{S)] A m and r{S) = r/q{S) . (5.1) 

Then we choose an index from the set As as 

where k is the parameter of the set and t„ = /„e. Finally, we set 

S = S-^ and ~X = \^. (5.2) 
Now we show the upper bound (j4.5p for this estimator. 
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Theorem 5.1. Assume that condition Hi) hold. Then 

limsup TZ^(S,S)/jk{S) <1. (5.3) 



Remark 5.1. Note that the estimator S belongs to estimate family 
(|2.4p . but we can't use directly this estimator because the parameters k, r 
and r{S) are unknown. We can use this upper bound only through the oracle 
inequality (j4.2|) proved for procedure (|2.8p . 

Proof. To prove the theorem we will adapt to the heteroscedastic case the 
corresponding proof from Nussbaum, 1985. 

First, from ()2.3p we obtain that, for any p £ V^,, 



n 1 " 



n 

j=i j=i 



where 

1 " 



n 
1=1 



Setting now Hi = to (a), jo = [uj/lnn], ji = [ujlnn] and 

1 " 

1=1 

we rewrite (15. 4p as follows 

ji-l n 

BsJS-S\\l= J2 (l-A,)'^,',„ + ?nn-ij; A| + Ai(n) + A2(n) (5.5) 
j=jo+l i=l 

with 

n n 

Ai(n) = and ^^{n) = n'^ J] (?,- , - ?J . 

j=ji 
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Note that we have decomposed the first term in the right-hand of (|5.4p into 
the sum 

(l-A,)X„ + Ai(n). 

i=io+i 

This decomposition allows us to show that Ai(n) is negligible and further 
to approximate the first term by a similar term in which the coefficients 'd^ ^ 
will be replaced by the Fourier coefficients '&j of the function S. 

Taking into account the definition of a; (a) in (j2.5p we can bound Co as 

~ > (^,)V(2^-+i) ^i/(2fc+i) (inn)-V(2A:+i) _ 
Therefore, by Lemma lA.ll we obtain 



lim sup n2'=/(2fc+i)^^(^) 
Let us consider now the next term A2(n). We have 



lA.fn) 



^ n n 



d=l j=l 



< — sup 

^ 0<x<l 



where (/>j (x) = (p'ji^) ~ 1- Now by Lemma [A. 21 and definition (|2.5|) we obtain 
directly the same property for A2(n), i.e. 



Setting 



lim sup ^0 



j=jo 



and applying the well-known inequality 



(a + 5)2 < (l + S)a^ + {l + l/6)b^ 
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to the first term in the right-hand side of inequality ()5.5p we obtain that, 
for any 6 > and for any p £ V^, 

EsJS-S\\l<il + 6)%^^{S)n-''/(''+'^ 

+ Ai(n) + A2(n) + (l + l/5)A3(n), (5.6) 

where 

A3(n)= 

i=jo+i 

Taking into account that A: > 1 and that 

ii < ^i/(2fc+i)(inn)(2fc+2)/(2^-+i) , 

we can show through Lemma lA.31 that 

hm sup n2'=/(2fc+i)^g(^) 

Therefore inequahty (|5.6|) yields 

limsupn2^/(2^+i) sup 7^„(^, 5)/7fc(5) < hmsup sup %,„(5)/7fc(5) 

and to prove (|5.3|) it suffices to show that 

limsup sup 7fc,„(5')/7fc(5) < 1. (5.7) 

First it should be noted that definition (|5.1|) and inequalities (|3.7p - 
imply directly 

lim sup |t„/r(5) — l| = . 
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Moreover, by the definition of (Aj)i<j<„ for sufficiently large n for which 
tn ^ ^(5*) we can calculate the following supremum 

sup n2'=/(2fc+i)(^ _ A,)V(^jf' = 7T-^'{A,iJ-''/^''+'^ 
i>i 

Therefore, taking into account the definition of the coefficients {0'j)j>i in 
p.Sp we obtain that 

limsupn2'=/(2'=+i) sup sup 7r2'=(^,r(5))''/(''+') (1 - A,)Va, < 1 . 

'^^oo S£W>y j>jo 

Moreover, by definition (j2.5p we get that 



lim sup 



l/{2fe+l) ^^2_(^^-(^)^l/(2fc+l) / (l_^fe)2d^ 
.7 = 1 ' 



n 

j 

k 



0. 



Taking into account definition of in (13. 3p and condition ()3.6p we obtain 
inequality (j5.7p . Hence Theorem 15. li □ 

Now Theorem 14.11 and Theorem 15.11 imply Theorem 14. 2[ 



6 Lower bound for parametric heteroscedastic re- 
gression models 

Let (M",e(M"),P^,T9 G 9 CM') be a statistical model relative to the ob- 
servations (yj)i<j<„ governed by the regression equation 

Vj = S^{xj) + 00)^^, (6.1) 

where ■^n are i.i.d. AA(0, 1) random variables, -d = . . . , -di)' is a 

unknown parameter vector, S^{x) is a unknown (or known) function and 
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(Tj(??) = g{Xj,S^), with the function g{x,S) defined in condition H^). As- 
sume that a prior distribution fi^ of the parameter i) in is defined by the 
density $(1?) of the following form 

I 

i=l 

where (pi is a continuously differentiable bounded density on R with 

M = I — < °° • 

Let A(-) be a continuously differentiable — > R function such that, for any 
\<i<l, 

lim A(7?)^^(7?J = and / |A'(??)| $(^?)d^? < 00 , (6.2) 

where 

>:,{^) = {d/dd,)m. 

Let A„ be an estimator of A(i?) based on observations (%)i<j<„. For any 
B{W^ X R') - mesurable integrable function 1?), x G R", i9 G R', we set 

EG(y,i?) = / E^G(F,i?)$(??)d7?, 

where is the expectation with respect to the distribution of the vector 
Y = {y-^, . . . ,y^). Note that in this case 

E^G(y,7?) = [ Giv,i»)f{v,i9)dv, 

where 

/(^,^) = fr ^ exp I- . (6.3) 

We prove the following result. 
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Theorem 6.1. Assume that conditions Hi) — H2) hold. Moreover, assume 
that the function S^{-) is uniformly over < x < 1 differentiable in C[0, 1] 
with respect to 1}^, 1 < i < I, i.e. for any 1 < i < / there exists a function 
S'^ ■ £ C[0,1] such that 

lim max (s^^f^^Xx) - S^{x) - S'^ .{x)h) /h =0, (6.4) 

where e^ = (0, 1, ...,0)', all coordinates are 0, except the ith equals to 1 . 
Then for any square integrable estimator A„ of X{'d) and any 1 < i <l, 

E(A„ - \f > Aj/{Fi + Bi+ I,) , (6.5) 

where A, = J^, X[{i^) F, = E"=i J^^ {S'^^,{xj) / a>(i?)d^ and 

Lj(x,'!9) = 5^(5^ J, the operator g is defined in the condition H2). 
Proof. We put 

Note that due to condition (j3.7p the density (|6.3p is bounded, i.e. 

fiv,^)<{27rg,)-^/\ 
So through (|6.2|) we obtain that 

hm mf{v,i))^,{^,)=0. 
Therefore, integrating by parts yields 

E(A„-A)^,= / cuv)-m)^{fiv,mmdMv 

Jmn+l OVi 



/..(^''"O H"'=*- 
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Now the Bouniakovskii-Cauchy-Schwarz inequality gives the following lower 
bound 

To estimate the denominator in the last ratio, note that 



t\" I) 



where 

f,iv,^) = id/d^.,)lnfiv,^). 
From ()6.ip it follows that 

Moreover, conditions H2) and ()6.4p imply 

id/d^i)a]{^) = d/d'&i)g\xj,S^) = L,(xj,^) 
from which it follows 

E (/,;(y,^?))' =F, + B,. 
This implies inequality (j6.5p . Hence Theorem 16. 1[ □ 

7 Parametric kernel function family 

In this section we define and study some special parametric kernel functions 
family which will be used to prove the sharp lower bound (|4.6p . 
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Let us begin by kernel functions. We fix 77 > and we set 

w = [ i(i«i<i-^) y (^) d« ' (7-1) 

JR \ 'I / 

where 1a is the indicator of a set A, the kernel V G C°°(M) is such that 

V{u) = for \u\ > 1 and J V{u) du = 1 . 

It is easy to see that the function Iri{x) possesses the properties : 

< < 1 , Ir,{x) = 1 for |x| < 1 - 2r? and 
Irjix) = for |x| > 1 . 

Moreover, for any c > and m > 1 



lim sup 

/:|/U<c 



/ /(x)/;X^)dx - /' f{x)da 

JR J-l 



0, (7.2) 



where |/|^ = sup_^<^<^ |/(x)|. 

We divide the interval [0, 1] into M equal parts of length 2h and on each 
of them we construct a kcrncl-type function that was used in Ibragimov, 
Hasminskii, 1981, to obtain the lower bound for estimation at a fixed point. 
A such constructed on each interval function equals to zero at the extrem- 
ities together with all derivatives. It means that Fourier partial sums with 
respect to the trigonometric basis in /^2[~l!l] S^^^ ^ natural parametric 
approximation to the function on each interval. 

Let {ej)j>i be the trigonometric basis in L2[— 1, 1], i.e. 



ei = I/V2 , ej{x) = Trj {7r[j/2]x) , j>2, (7.3) 
where Trj{x) = cos(x) for even j and Trj{x) = sin(a;) for odd j. 
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Now, for any array z = {(-2mj)i<m<Af ,i<j<Af } define tlie following 
function 

Sz,ni^) =^Y1 ^rn,j Dm,j{x) , (7.4) 
m=l j=l 

where D^ j{x) = ej {vm{x)) Ir, (vmix)), 

Umix) = (x - Xm)/K, Xm = 2mh^ and M„ = [l/(2/i„)] - 1 . 

We assume that the sequences (-/V^)„>i and (/i„)„>i, satisfy the following 
conditions. 

Ai) The sequence — > oo as n —> oo and for any p > 

lim N^/n = 0. 

n—*oo 

Moreover, there exist < 5i < 1 and S2 > such that 

h^=0{n~^^) and = 0{n^^) as n ^ 00 . 

To define a prior distribution on the family of arrays, we choose the following 
random array i9 = {{^m,j)i<m<M^ ,i<j<Nj with 

where {(mj) i.i.d. 7\A(0, 1) random variables and {tm,j)i<m<M^ ,i<j<N^ 
are some nonrandom positive coefficients. We make use of gaussian variables 
since they possess the minimal Fisher information and therefore maximize 
the lower bound ()6.5p . We set 

t* = max t , . (7.6) 
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We assume that the coefficients {tm,j)i<m<M ,i<j<N satisfy the following 

conditions. 

A2) There exists a sequence of positive numbers ((i„)„>i such that 

li- TO E E ^'^'-'^ = ' lim v^C = > (7-7) 

m=l j=l 

moreover, for any p > 0, 

lim n*' exp{-d„/2} = . 

n— >oo 

A3) For some < e < 1 

" m=l j=l ^ 

A4) There exists eg > such that 

lira y y =0. 

m=l j=l 

Proposition 7.1. Lei conditions A^)-A2). Then, for any p > and for 

any 5 > 0, 

lim n*' max P (\\s!Ii\\ > d) = . 

n->oo 0<Kik-l V ) 

Proof. First note that for < a; < 1 we can represent the Zth derivative as 

= 77 E E 0) Qi.miv^^^)) , (7.8) 

m=l 1=0 
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where 

j=i 

Therefore 



\stf = sir E / E (0 O.^.W 

1^ m=l 1 \i=0 / 



and by the Bounyakovskii-Cauchy-Schwarz inequahty we obtain that 



c.(0 ||2 < £U^V- O (7()\ 



1=0 



With C*(/,r?) = max_i<„<i ((D and 

Q. = y / q2 / ) ^ _ 

m=l 

Now we show that for any < i < A; — 1 and (5 > 

l™^'Pfe,..>'^^rO=0- (7-10) 

n— >oo \ ' / 

To that end we introduce the following set 

S = { max max ^ < t^nl ' ('''■11) 

where the sequence ((i„)„>i is given in condition A2). Therefore, taking into 
account that 
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the function Q • ^ can be estimated on the set H„ as 

Q. < (ly d^yyt^ .fi 

m=l j=l 

and by (17.70 we get, for any 6 > and sufficiently large n, 

P fe. > Sh^''-^) < PfH-^) . 
Moreover, for sufficiently large n 

P(H^)< M„iV„e-'^"/^ 
Therefore, conditions A^^) and (|7.7p imply 

limsup nPp (H^) = 0, (7.f2) 

n— »oo 

for any p > Q. Hence Proposition 17.11 
□ 

Proposition 7.2. Let conditions Then, for any p > 0, 

lim nPP{S^^^W,^) =0. 

n— >oo ' 

Proof. First of all we prove that for e from condition A3) 

lim nPP (\\sfl\\ > V(l - e/4)r) =0. (7.13) 

n— >oo V ' / 

Indeed, putting in (j7.8p I = k we can represent the fcth derivative of 5*^ ,^ as 
follows 

^£(^)=4(2^) + 5fc(x) (7.14) 

with 

M h—^ 

^ki^) = E E (') 4'"^^(^rn(^))Q.m(^^(^)) 
m=l i=0 
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and 

1 ^" 

m=l 

First, note that, we can estimate the norm of 5'^(x) by the same way as in 
inequaUty (I7.9p . i.e. 

By making use of (j7.10p we obtain that, for any p > and for any S > 0, 

hm nPP fll^fcll > 5) =0. (7.15) 



Let us consider now the last term in ()7.14p . Taking into account that 



< Ir,{v) < 1 we get 



11"^ l|2 ^ 



m=l ^ 



m=l j=l 

Therefore from condition A3) we get for sufficiently large n 

M 

\\S,f<il-s/2)r+[-) E - (1 - ^/2)- + ( 2 ) 

with 

1 

7 - V f2 i2fc ? and - /-2 _ -, 

~ /l2fe-l m,jJ ^rn,j ^^^"-^ W,j — i^mj ' 

We show that for any p > and for any 6 > 

lim nPP{\Yj >6) = 0. (7.16) 

n— >oo 

Indeed, by the Chebyshev inequality for any z. > 

P{\Yj>6)<B{Yj'/6^\ (7.17) 
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Note now that according to the Burkholder-Davis-Gundy inequahty for any 
t > 1 there exists a constant B*{l) > such that 



i m=l 

Moreover, by putting 

C = max max P . 

l<m<M„ l<j<N^ 

we obtain that 

AT 

— ^4fc— 2 / y m,j ^* ' 

Therefore, by condition A^) for sufficiently large n 

E(yj''<i?*(OK/i::°Ec: 

< B*{i) E (C' - if M^N'+^h'^o , 



where C ~ -^(0, !)• Taking into account here condition Ai) we obtain for 
sufficiently large n 

E {YJ^' < n~^^ (^^0-2) . 

Thus, choosing in (j7.17p 

i > P/(eo'5i) + 2/eo 

we obtain limiting equality (j7.16p which together with (|7.14p - ()7.15p implies 
(|7.13p . Now it is easy to deduce that Proposition 17.11 yields Proposition I7.2[ 
□ 

Proposition 7.3. Let conditions A'i^)-A^). Then, for any p > 0, 
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Proof. First of all, we remind that due to condition A2) 

lim yyf .< lim yyt' .f^'^-^^ = o. 

n— >oo '-^ n^oo ilzy — — 

ra=\ j=\ " m=l j=l 

Therefore, taking into account that 
we obtain, for sufficiently large n. 

Moreover, for any 1 < m < M„ and 1 < j < A^„, we estimate the last term 



as 



where C ~ AA(0, 1). By applying now Proposition 17.21 and limit (|7.12|) we 
obtain Proposition 17.31 □ 

Proposition 7.4. Let conditions Ai)-A^). Then for any function g satis- 
fying conditions (j3.7p and H4) 

lim sup E I 5"^(x,5'^_„) - = 0. 

Proof. First, note that on the set H the random function S^^^ is uniformly 
bounded, i.e. 

\S&J*= sup |5^„(x)| < v^t* , (7.19) 

0<x<l 
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where the coefficient i* is defined in ()7.6p . Therefore by condition Hj^) we 
obtain 

¥.\g-\x,S^^^)-g^\x)\< max \g-\x,S)-g~\x)\ + {'^l g,)V {r.^) . 

Conditions Ag) and H4) together with the hmit relation ()7.12|) imply Propo- 
sition [731 □ 

8 Lower bound 

In this section we prove Theorem l4.31 To that end we establish the following 
auxiliary result. 

Lemma 8.1. For any < 5 < 1 and any estimate Sn of S £ , 

\\Sn-S\\l>{l-5)\\T^{S)-Sf - (5-1 - l)r/n\ 
where T^{S){x) = Y2=i Sn{xk)'^(^^^_^^^^^{x). 

Proof of this Lemma is given in Appendix IA.2I 

This Lemma implies that to prove (j4.6p . it suffices to show the same 
asymptotic inequality for the integral risk, i.e. 

liminf inf n2^V(2fc+i) n^iSn) > 1 , (8.1) 

n— >oo Q 

where 

^o(5'n) = sup ^s,q\\^n- SW'^ /-fk{S) , 

q is the gaussian (0, 1) density of the noise (^j) and \\S\\'^ = S^{x)dx. 
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To show ()8.ip we will make use of the sequence of random functions 
i'^^,n)n>i defined in (j7.4p - (|7.5p with the coefficients (imj) satisfying condi- 
tions Ai)-A^) which will be chosen later. 

For any estimator Sn, we denote by its projection onto W^^, i.e. 

= Fi'^Yk{Sn)- Since W^^ is a convex set, we get that 

HQ Qll^ ^ II QII^ 
II — 1 1 II 

Therefore, we can write that 

f E5 g\\S^-S,J^ 

Tlo{Sn)> / — — — — fJ-ffidz). 

Here denotes the distribution of tl) in with / = M^N^. We recall also 
that the set H„ is defined in (j7.1ip . Moreover, taking into account here 
inequality (|7.19p we estimate the risk TZQ{Sn) from below as 

with 

7:= sup ^u{S). (8.2) 

|sL<^t; 

Let us introduce now the corresponding Bayes risk 

Ms'i)=! ^s^J\s'i-s,^S^l^o{dz). (8.3) 

Now through this risk we rewrite the lower bound for lZQ{Sn) as 

n^{sl)>no{sl)hi-2wj^l (8.4) 

with 
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First of all, we reduce the nonparametric problem to parametric one. For 
this we replace the functions and S by their Fourier series with respect 
to the basis 

By making use of this basis we can estimate the norm \\S^ — S^^P from 
below as 

m=l j=l 

where 

^^m,j= [ S^ix)em,j{x)dx and A (z) = / S"^ „(x)e^ -(x) dx . 
Jo Jo 



Moreover, from definition (j7.4p one gets 



ei{u)ej{u)Irj{u) du . 



It is easy to see that the functions Xmji') satisfy condition (j6.2p for gaussian 
prior densities. In this case (see the definition in ()6.5p ) we have 



where 

e,(/) = f^e]{v)f{v)dv. (8.5) 

Now to obtain a lower bound for the Bayes risk 1Zq{S^ we make use of 
Theorem 16. II which implies that 

m=l j=l ™J ~'~ ~'~ m.i 
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where = ^^=1 Dl/xi)E g-^x„ S^^J and 



1 " 



i=l 



with L^j(x, 5) = ^5 (L'^ j). In the appendix we show that 



hm sup sup \F^ j/{nh) - ej{I^) {xm)\ = 

n^oo l<m<M„ l<j<N^ 



and 



hm sup sup \B,^j/{nh)\ = 0. 
This means that, for any > and for sufficiently large n, 



sup sup 



771 -L R -L f~ 

m.j ' m,j ' m,j 



Therefore, if we denote in (18.61) 



<l + u. 



(8.7) 



(8.8) 



n/i^o '\xm) and Tj(r?, y) = e'^j{I^) y/{ej{^)y + 1) 



we obtain that, for sufficiently large n, 
In the appendix we show that 



m=l 



lim sup sup 

ri^O 7V>l(y^,...,y^)eRiv 



(8.9) 



where 



T"(y) = y/{y + 1) 



Therefore we can write that, for sufficiently large n, 

2k ~ 1 — 1/ 1 r — ^ „ o 



1 + 



(8.10) 



m=l 
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where 

N 

Obviously, to obtain a "good" lower bound for the risk 72.o(5'n) one needs 
to maximize the right-hand side of inequality (jS.lOp . Hence we choose the 
coefficients (/^^j) by maximization of the function J^, i.e. 

N 

max Jj^{y^,...,y^) subject to y.f' < R . 
yi,-,yN -, 

The parameter i? > will be chosen later to satisfy condition A3). By 
the Lagrange multipliers method it is easy to find that the solution of this 
problem is 

N N 

y*{R) = {R + Y^f^)r'^/Y^j'^-l for l<i<iV. (8.11) 

3=1 j=i 

To obtain a positive solution in (|8.1ip we need to impose the following 
condition 

TV TV 

R> N'^Y.^'-T.^''- (8-12) 

j=l j=l 

Moreover, from condition A3) we obtain that 

R<2^^+\l-e)rnhf-^'/{7T^'g,) :=R*^, (8.13) 

where 

M„ 



% = 2/in Y 9l{Xm) 



m=l 



Note that by condition H4) the function 5'o(') = di'^^o) continuous on 
the interval [0,1], therefore 

hm 5o = gHx, So)dx = ?(So) (8.14) 



32 



with Sq = 0. 

Now we have to choose the sequence Note that if we put in (j7.5p 



i9oixni)/y/nh^) \Jy*j{R) i-e- k^j = y*iR) , (8.15) 
we can rewrite inequahty (j8.1(jp as 

n^7^o(50) > ^ ^ , (8.16) 

where 

/ TV \ 2 TV 
J^(i?)=iV- /{R + Y^j^^). 



It is clear that 



k^/{k + if < hminf inf J*^{R)/N < hmsupsup J*^{R)/N < 1 . 

N^oo R>0 N-^oo R>0 

Therefore to obtain a positive finite asymptotic lower bound in (j8.16p we 
have to take the parameter as 

^ = /i,n-i/(2^'+i)iV„ (8.17) 

with some positive coefficient h^. Moreover, conditions (18.12p - ()8.13p imply 
that 

f 1 _ £)r - /i2fe+i > ^ V i'' — V 



^ ,2k 



n „_i n 



Taking here hmit as n ^ oo thanks to asymptotic equality (j8.14p , we obtain 
the following condition on 

K > (t;*)V(2fc+i) ^ (818) 

where 

k - - e)r 

^ c*(A; + l)(2A: + l) - ^2fc^(5o) ■ 
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To maximize the function J'L (R) at the right-hand side of inequahty (|8.16p 

n 

we take R = i?* defined in (j8.13p . Therefore we obtain that 

hminf inf n''/^''+'^noiS'j > (?(5o)/2) F{K) , (8.19) 

n— >oo gO 



where 



p. ,1 2k + 1 

X (A; + l)2(c*(2A; + l)x2^+2 + x) ■ 



Taking into account that 



^"^^ (A: + l)2(c*(2A: + l)x2fc+2 + x)2 - 



we find that 



max F{K) = F{{v;y/(''+'y) = {k/{k + l))(z;:)-V(2'=+i) . 

h,>(w*)l/(2fe+l) 

This means that to obtain in ()8.19p the maximal lower bound we have to 
take in (fSTTll 

= (^;*)i/(2fc+i) . (8,20) 
Therefore, inequality (|8.19p implies 

hminf inf n^'^/^^'^+'^TZoiS^J > (1 - 5)^(2^+1) ^,(5^) , (§.21) 

■n—*cx) qO 

where the function 7fc(5'o) is defined in (j4.4p for Sq = 0. 

Now to end the definition of the sequence of the random functions {S^ „) 
defined by (j7.4p and (j7.5p we have to define the sequence (N^)- We remind 
that we make use of the sequence (5'^_„) with the coefficients (t^ j) con- 
structed in (j8.15p for R = i?* given in (j8.13p and for the sequence given 
by (j8.17p and (j8.20p for some fixed arbitrary < e < 1. 
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We will choose the sequence (A''„) to satisfy conditions A;^)-A4). We can 
take, for example = [In^ n] + 1. Then condition A-^) is trivial. Moreover, 
taking into account that in this case 

^2kg^ e n 1)(2A; + 1) " 

we find thanks to convergence (|8.14p 

(K + Ej''')/(^nEj'') = l- 

Therefore, solution (jS.lip for sufficiently large n satifies the following in- 
equality 

max y*{R*)f < 2N^ . 

Now it is easy to check conditions A2) with (i„ = \/N^ and A4) for arbitrary 
< Eq < 1. As to condition A3), note that by definition of t^j in (jS.lSp we 
have 



_ „-2/c 

hi''- 

m=l j=l j=l 

2k 



_L \ ^ \ " ,2 ■2k _ ^ " \ " *fn*\ 

ik-\ Vi-^ ~ 2n/i2fc+i ° ^ yj^^J^ 



n e ^ 



Hence condition A3). 



Therefore Propositions 17. 2117. 31 and limit ()7.12p imply that for any p > 



lim TiP = . 

n— >oo 



Moreover, by condition H4) the sequence 7* goes to 7fc(5'o) as n — > 00. 
Therefore, from this, (I8.21|) and (18. 4p we get for any < e < 1 

hminf inf n^^/^^^+i) 7^o(5„) > (1 - e)i/(2^'+i) . 

Limiting here e — > implies inequality (18. ip . Hence Theorem 14.31 □ 
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9 Appendix 



A.l Properties of trigonometric basis 



Lemma A.l. For any function S G W^', 



sup sup m 

ra>l l<m<n— 1 



2k 



vj=m,+l 



< 



4r 



IT- 



2(k-l) 



Lemma A. 2. For any m > 0, 



sup sup N ™ 

A^>2 a:e[0,l] 



N 



1=2 



< 2'^ 



(A.l) 



(A.2) 



where (piix) = </)^(x) — 1. 



Proofs of Lemma lA.ll and Lemma IA.2I are given in [6] . 

Lemma A. 3. Let 9j „ and 9j be the Fourier coefficients defined in (12.20 and 
(|3.4p respectively. Then, for 1 < j < n and n>2, 



S&W} 

Proof. Indeed, we have 



sup 16*^ ,^ — 0^1 < 27r ^/rj/n. 



(A.3) 



E/ {S{xi)(j)j{xi) - S{x)(j)j{x))dx 
1=1 -'^i-i 

^ E r + \siz)Mz)\) dz 

1=1 -'^l-l 

= n-i {\S{z)\\c^,{z)\ + \S{z)\\<Pj{z)\)dz. 
By making use of the Bounyakovskii-Cauchy-Schwarz inequahty we get 



< n 



-1 



1511 



\S\ 



< n-^ (^\\S\\ + TTj \\S\\j . 
The definition of class Wj^ imphes (1A.3P . Hence Lemma lA.ll □ 
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A. 2 Proof of Lemma 18.11 

First notice that, for any S G W^, one has 

= ||r„(5)-5f + M/„ + A„, 

where 

'^n = 2Y^ P {Sn{Xj)-S{x)){S{x)-S{Xj))dx 
j=l ^j-i 

and 



^n = Yl r {S{x)-S{Xj)fdx. 

1 .7! 1 



For any < (5 < 1, by making use of the elementary inequaUty 

2ah < 5a^ + 5'^}? , 

one gets 

^'n < 5\\T^{S)-Sf + 5-^^n. 

Moreover, for any S G W^' with A; > 1, by the Bounyakovskii-Cauchy- 
Schwarz inequahty we obtain that 



n ^ — ' / 71^ 

j=i ^j-i 

Hence Lemma l8.ll □ 
A.3 Proof of (EZD 

First of ah, note that Proposition 17.41 condition l\3.7\) and condition H4) 
imply that 

lim max sup l{\^^^^)\<^yB\g~'^{x, ) - 5o^(5m)| = 0. (A.4) 

n^oo l<m<Af„ o<x<l 
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0. (A.5) 



Let us show now that for any continuously difFerentiable function / on [—1,1] 

n 1 

hm sup — )|<i| - / f{v)dv = 

— — n 1=1 

Indeed, setting 

1 " 

^n,m = ^Y1 fM^i))'^{\vM\<l} - J /(^)d^ 
i=l ~^ 

we obtain that 



A.. 



|/(z;„(x,)) - f{z)\dz + max |/(z)|(2 - v* + 



z <1 



where = [nx^ — nh] + 1, = [nx^ + n/i], 

t;^ = {[nx^ — nh] + 1 — nx^)/{nh) and v* = {[nx^ + nh] — nx^)/{nh) . 

Therefore, taking into accout that the derivative of the function / is bounded 
on the interval [—1, 1] we obtain that 

|A„ I < 3 max \f{z)\/{nh J + 2 max \f{z)\/ (nh J . 



\z\<l 



z\<l 



Taking into account the conditions on the sequence (/i„)„>i given in A^) we 
obtain limiting equality (jA.SP which together with ()A.4p implies (j8.7p . □ 



A. 4 Proof of 



Now we study the behaviour of B^ j . Due to inequality (j3.9p we obtain that 



|L^j(x, < C* {\S^^^{x)Dj^j{x)\ + I-Dmjii + ll'S',j,nll 11-^' 
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Note that 



, 1=1 



1=1 

We remind that the sequence t* is defined in (j7.6p . Therefore, property 
()A.5p implies 

1 " 

max max — V E{S^^{xi)D^ Axi)f = 0{{t*f) . 
l<m<M„ i<i<Af„ nh ^ ' " 

1=1 

Moreover, as to the function •(•) we find that 



\D^.j\i = [ \ej{Vmix)) Ir,{v^{x))\dx = h I \ej{v)I^{v)\dv < 2h . 



Similarly we obtain — ^• 

Finally, by (j7.18|) we obtain that 

m=l j=l 

Therefore, 

max max B^Jinh) = 0((t*)^ + /i„) 

l<m<M„ l<j<N^ " 

and condition A^) implies (|8.8p . 
□ 

A. 5 Proof of (ED 

Indeed, by the direct calculation it easy to see that for any > 1 and for 
any vector {y^, . . . , y^Y G 1^+ 



< 



min^.>i ej{P) 
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where the operator ej{f) is defined in in (|8.5p . Moreover, we remind that 
f^_^e'^{v)dv = 1. Therefore, taking into account property (j7.2p we obtain 
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